Background and Foreground Modelling Using an Online EM Algorithm

A novel approach to background/foreground segmentation using an online EM algorithm is presented. The method models each layer as a Gaussian mixture, with local, per pixel, parameters for the background layer and global parameters for the foreground layer, utilising information from the entire scene when estimating the foreground. Additionally, the online EM algorithm uses a progressive learning rate where the relative update speed of each Gaussian component depends on how often the component has been observed. It is shown that the progressive learning rate follows naturally from introduction of a forgetting factor in the log-likelihood. To reduce the number of mixture components similar foreground components are merged using a method based on the Kullback-Leibler distance. A bias is introduced in the variance estimates to avoid the known problem of singularities in the loglikelihood of Gaussian mixtures when the variance tends to zero. To allow a decoupling of the learning rate of the Gaussian components and the speed at which stationary objects are incorporated into the background a CUSUM detector is used instead of the prevailing method that uses the ratio of prior probability to standard deviation. The algorithm is scale invariant and its properties on grayscale and RGB videos, as well as on output from an edge detector, is compared to that of another algorithm. Especially for the edge detector video performance increases dramatically.